Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: build the Docker Driver for arm64 #9247

Merged
merged 5 commits into from
Nov 15, 2024

Conversation

tucksaun
Copy link
Contributor

@tucksaun tucksaun commented Apr 23, 2023

What this PR does / why we need it:
Add ARM64 build and release of the Docker driver in Drone pipeline

Which issue(s) this PR fixes:
Fixes #5682

Special notes for your reviewer:
I would have loved to have a unified x64 and arm64 build but apparently Docker drivers does not support multi arch images. So instead I went with tweaking the build steps to allow cross building the image for ARM64 and added the instructions to do so in drone.yml.
It seems there's a drift in the images published on Docker Hub versus the one documented for use. I updated it here but this might be wrong.

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated (no tests for this AFAIK)
  • CHANGELOG.md updated
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/upgrading/_index.md

@tucksaun tucksaun requested review from JStickler and a team as code owners April 23, 2023 20:48
@CLAassistant
Copy link

CLAassistant commented Apr 23, 2023

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Apr 23, 2023
@tucksaun tucksaun force-pushed the docker-driver-amd64 branch 2 times, most recently from 20e05b7 to d9cac9a Compare April 23, 2023 20:52
@jeschkies
Copy link
Contributor

@tucksaun could you resolve the conflicts?

.drone/drone.yml Outdated
@@ -1500,6 +1500,7 @@ name: docker-driver
steps:
- commands:
- make docker-driver-push
- make docker-driver-push PLUGIN_ARCH=-arm64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to change the jsonnet file uinstead. I'll generate the Drone YML then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jeschkies a friendly ping to let you know this PR has been rebased but requires your intervention for the .drone/drone.yml generation 🙂

@tucksaun tucksaun force-pushed the docker-driver-amd64 branch from d9cac9a to 4d7a281 Compare September 9, 2023 06:37
@pull-request-size pull-request-size bot added size/S and removed size/M labels Sep 9, 2023
@tucksaun tucksaun force-pushed the docker-driver-amd64 branch 2 times, most recently from 1898680 to a1b1ecc Compare September 9, 2023 06:40
@tucksaun
Copy link
Contributor Author

tucksaun commented Sep 9, 2023

@jeschkies conflicts resolved

@guvmao
Copy link

guvmao commented Oct 5, 2023

Is there any issue preventing the advancement of this pull request?

@tucksaun
Copy link
Contributor Author

tucksaun commented Oct 5, 2023

Is there any issue preventing the advancement of this pull request?

there were new conflicts I just resolved. excluding those, I believe someone (@jeschkies?) has to regenerate the drone.yml now that the jsonnet file is updated.
and that's it?

@tucksaun tucksaun changed the title Build the Docker driver for arm64 build: build the Docker Driver for arm64 Jul 20, 2024
@tucksaun tucksaun changed the title build: build the Docker Driver for arm64 feat: build the Docker Driver for arm64 Jul 20, 2024
@tucksaun tucksaun force-pushed the docker-driver-amd64 branch 2 times, most recently from 5a31800 to f7c1227 Compare July 26, 2024 07:47
@jeschkies
Copy link
Contributor

@tucksaun I'm sorry. I've switched teams and have missed GitHub notifications.

I would have loved to have a unified x64 and arm64 build but apparently Docker drivers does not support multi arch images.

I was about to mentioned the multi arch image. How is the arm image tagged differently?

@tucksaun
Copy link
Contributor Author

tucksaun commented Aug 22, 2024 via email

@francescor
Copy link

Hi, I think your PR would be very helpful

is it a lot of work to solve conflicts to this PR can be accepted?

@tucksaun tucksaun force-pushed the docker-driver-amd64 branch from f7c1227 to 288105a Compare October 4, 2024 07:55
@tucksaun
Copy link
Contributor Author

tucksaun commented Oct 4, 2024

@francescor not much for the Dockerfile so this is done.

however this is a bit different for the pipeline because Drone has been recently removed (see #14273) and it seems like the docker plugin is neither build not released anymore (I can't find any reference to it anyway).

@jeschkies, this means I'm not able to update the pipeline for testing or releasing anymore 🤷

@tucksaun tucksaun requested a review from jeschkies October 4, 2024 07:57
@francescor
Copy link

I see this one https://hub.docker.com/r/miacis/loki-docker-driver

is related to this context?

@tucksaun
Copy link
Contributor Author

tucksaun commented Oct 4, 2024

I see this one https://hub.docker.com/r/miacis/loki-docker-driver

is related to this context?

I have no idea

@trevorwhitney
Copy link
Collaborator

Naming should be be grafana/loki-docker-driver which is how it used to be published. As for a template, you should be able to use the jsonnet functions to just add another image job in release-workflows.jsonnet and then run make release-workflows. thanks!

**What this PR does / why we need it**:
Add ARM64 build and release of the Docker driver in Drone pipeline

**Which issue(s) this PR fixes**:
Fixes grafana#5682

**Special notes for your reviewer**:
I would have loved to have a unified x64 and arm64 build but apparently Docker drivers does not support multi arch images.
So instead I went with tweaking the build steps to allow cross building the image for ARM64 and added the instructions to do so in `drone.yml`.

**Checklist**
- [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**)
- [x] Documentation added
- [ ] Tests updated
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
@tucksaun
Copy link
Contributor Author

tucksaun commented Nov 7, 2024

@trevorwhitney this is added. But this required more changes than you probably anticipated (Docker plugins are not shipped as images...).
Also the use of jsonnet does not make it easy to try the changes locally so please be thorough during review :)
I added it at a second commit to make it easier for you to review it.
I also extracted the changes to a PR to https://github.com/grafana/loki-release: grafana/loki-release#162

Copy link
Collaborator

@trevorwhitney trevorwhitney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome, thank you for tackling this! unfortunately we can't edit the vendored jsonnet directly at that will get wiped out. that lives in grafana/loki-release, so we'll need to port the changes over there. in the process, is there any existing GitHub action for creating the docker plugin, or do we have to do it by hand? thanks!

'build-args': 'IMAGE_TAG=${{ needs.version.outputs.version }},GOARCH=${{ steps.platform.outputs.platform_short }}',
}),

releaseStep('Package as Docker plugin')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there an existing github action that handles the creation of docker plugins we can use?

@tucksaun
Copy link
Contributor Author

tucksaun commented Nov 8, 2024

awesome, thank you for tackling this!

you're welcome 🙂

unfortunately we can't edit the vendored jsonnet directly at that will get wiped out. that lives in grafana/loki-release, so we'll need to port the changes over there.

yes this why I also extracted the changes and opened grafana/loki-release#162

in the process, is there any existing GitHub action for creating the docker plugin, or do we have to do it by hand? thanks!

not that I know of

Copy link
Collaborator

@trevorwhitney trevorwhitney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks for making the loki-release change as well

@trevorwhitney trevorwhitney merged commit e950251 into grafana:main Nov 15, 2024
58 checks passed
@francescor
Copy link

Super!
The official grafana/loki-docker-driver:latest works in ARM without the emulator!

Thank you very much to everybody, expecially tucksaun of course!

@trevorwhitney
Copy link
Collaborator

yes, indeed, big credit to @tucksaun, thanks for the contribution!

@tucksaun tucksaun deleted the docker-driver-amd64 branch November 16, 2024 11:23
thevops pushed a commit to thevops/loki that referenced this pull request Nov 21, 2024
Co-authored-by: Trevor Whitney <trevorjwhitney@gmail.com>
@francescor
Copy link

Hello!
I've just created a new node and official ARM plugin does not work,

(I had to go back to @tucksaun 's docker plugin install tucksaun/loki-docker-driver:main-arm64 --alias loki --grant-all-permissions)

how to nodify developers? should I open a new issue?

@trevorwhitney
Copy link
Collaborator

@francescor yes, please create an issue with as much detail as you can provide so someone can recreate the problem.

@francescor
Copy link

I 've just created the issue #15318

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/L type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Docker Loki Driver ARM64
6 participants